
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 223 13-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


09/964,947 


09/26/2001 


James Chi-Shun Tsiao 


490.11 


3725 



s 1 



8685 7590 05/13/2005 

DERGOSITS & NOAH LLP 

FOUR EMBARCADERO CENTER, SUITE 1450 

SAN FRANCISCO, CA 941 1 1 



EXAMINER 



SKED, MATTHEW J 



ART UNIT 



PAPER NUMBER 



2655 

DATE MAILED: 05/13/2005 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 


Application No. 

09/964,947 


Applicant(s) 

TSIAO ET AL. 


Pvaminor 

examiner 
Matthew J Sked 


Art Unit 

2655 





The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )K Responsive to communication(s) filed on 27 January 2005 . 
2a)E3 This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) IEI Claim(s) f ,2,4,5, 7-10, 12 and 26-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) IEI Claim(s) 1,2.4,5.7-10.12 and 26-28 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date . 6) O Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 20050509 



Application/Control Number: 09/964,947 Page 2 

Art Unit: 2655 

DETAILED ACTION 
Response to Amendment 

1 . The objection to the disclosure and claims is withdrawn in view of the 
applicant's arguments. 

2. The objection to claims 13 and 16 under 37 CFR 1 .75 (c) is withdrawn in 
view of the applicant's amendments. 

3. Claims 1 and 26 have been amended to more clearly claim the invention 
as a handheld personal assistant that categorizes data in a database for retrieval 
based on both voice recognition and natural language understanding to 
determine who is speaking and the context of the spoken query. 

4. Claims 3, 6, 1 1 , 13-25 and 29 have been cancelled. 

5. Applicant's arguments with respect to claims 1 , 2, 4, 5, 7-10, 12 and 26-28 
have been considered but are moot in view of the new ground(s) of rejection. 

Claim Objections 

6. Claims 1 and 9 are objected to because of the following informalities: 
In claim 1, the third limitation should be changed from "a context 

processor configured determine" to -a context processor configured to 
determine-. 

In claim 9, the "a" after "includes" in the second limitation needs to be 
deleted. 

Appropriate correction is required. 
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Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
the treaty defined in section 351 (a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

8. Claims 1, 2, 4, 9 and 10 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Kanevsky et al. (U.S. Pat. 6,434,520). 

Kanevsky teaches a handheld personal assistant operated by a user 
comprising: 

a categorizer configured to store a plurality of data items in a database 
organized by one or more categories of information (indexes audio segments 
based upon background noise, identity of speaker and content, Fig. 2A and 2B); 

a voice-recognizer configured to recognize the user's voice and transform 
an expression input by the user into a different mode of information (generates a 
spoken voiceprint of the user, col. 9, lines 9-17); 

a context processor configured to determine the category of information 
corresponding to the expression input by the user, based on one of a direct 
context specified by the user or an inferred context based on the expression 
input (performs natural language processing for determining the context in order 
to search user queries, col. 7, lines 11-15); and 
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a natural-language processor configured to process the mode of 
information to extract, from the database, a piece of information that is personal 
to the user, wherein upon recognition of the user's voice, the personal assistant 
only allows the user to access the piece of information that is personal to the 
user (retrieves segments according to the natural language topic and speaker's 
voiceprint, col. 9, lines 9-17 and col. 9, line 60 to col. 10 line 9), 

and further wherein the natural-language processor can still extract the 
piece of information when the user declares the expression differently (natural 
language understanding allows recognition of expressions declared differently, 
col. 9, line 60 to col. 10, line 9). 

9. Regarding claim 2, Kanevsky teaches the processor analyzes the 
expression grammatically and semantically to transform at least part of the 
expression into at least one instruction (natural language understanding analyzes 
text both grammatically and semantically, col. 6 line 62 to col. 7, line 20). 

1 0. Regarding claim 4, Kanevsky teaches that the processor can still extract 
the piece of information even if the expression is ambiguous (natural language 
understanding determines the context hence determining the meaning of a word 
that would be ambiguous, col. 6 line 62 to col. 7, line 20). 

11. As per claim 9, Kanevsky teaches: 

the piece of information was entered into the assistant by the user (audio 
stream may be conversational speakers, col. 2, lines 57-66); and 

wherein the personal assistant further includes a formatting process that 
transforms the expression input by the user into a question with one or more 
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phrases corresponding to the one or more categories of information, and a 
transformation process the converts the question into an instruction comprising a 
query to the database (natural language understanding unit determines the issue 
or topic of the query and uses this issue to generate a search, col. 7, lines 1 1 - 
20). 

12. As per claim 10, Kanevsky teaches the piece of information was entered 
through voice (audio stream may be conversational speakers, col. 2, lines 57- 
66). 

Claim Rejections - 35 USC § 103 

13. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

14. Claims 1 , 2, 4, 5, 7-10, 12 and 26-28 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Sasai in view of Alpdemir. 

Sasai teaches a handheld personal assistant operated by a user 
comprising: 

a categorizer configured to store a plurality of data items in a database 
organized by one or more categories of information (private information storage 
unit stores the information in association with the user intention index and 
conceptual index hence categorizing the data, col. 6, lines 24-34 and Fig. 5); 
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a context processor configured to determine the category of information 
corresponding to the expression input by the user, based on one of a direct 
context specified by the user or an inferred context based on the expression 
input (concept extraction unit and intention interpretation unit process the text 
through natural language processing hence inferring the context of the input, col. 
4, line 45 to col. 5, line 1 5); and 

a natural-language processor configured to process the mode of 
information to extract, from the database, a piece of information that is personal 
to the user (the information management unit performs retrieval of data from 
private information storage unit, col. 5, lines 16-30), 

and further wherein the natural-language processor can still extract the 
piece of information when the user declares the expression differently (user may 
enter a sentence or a sequence of key words, col. 9, lines 46-51). 

Sasai does not teach a voice-recognizer configured to recognize the 
user's voice and transform an expression input by the user into a different mode 
of information; wherein upon recognition of the user's voice, the personal 
assistant only allows the user to access the piece of information that is personal 
to the user. 

Alpdemir teaches a PDA that has natural language speech recognition 
capabilities that suggests using voice recognition to authenticate a user (col. 6, 
lines 14-20). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the system of Sasai to authenticate a user for access to the 
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user's personal information as taught by Alpdemir because it would make the 
user's information more secure hence making the user more likely to store the 
information on the device. 

1 5. Regarding claim 2, Sasai teaches the processor analyzes the expression 
grammatically and semantically to transform at least part of the expression into at 
least one instruction (sentence structure analysis and semantic analysis, col. 4, 
lines 45-50). 

16. Regarding claim 4, Sasai teaches that the processor can still extract the 
piece of information even if the expression is ambiguous (presents plural pieces 
of information if conditions are not completely met, col. 10, lines 41-46). 

17. Regarding claim 5, Sasai suggests the piece of information selected from 
a list consisting of a personal address book (information is name and phone 
number, col. 6, lines 30-34) and a to-do-list (list of entries on display, Fig. 1 , 
element 3-2). 

Sasai and Alpdemir do not teach the information to be a calendar. 

However, the Examiner takes Official Notice that calendars are common in 
PDA's and it would have been obvious to one of ordinary skill in the art at the 
time of invention to modify the system of Sasai and Alpdemir for the information 
to be a calendar because it allows the user a simple way to keep track and 
monitor future appointments. 

1 8. As per claim 7, Sasai teaches a display to display the piece of information 
(Fig. 1, element 3-2). 
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1 9. As per claim 8, Sasai teaches a voice synthesizer that transforms the 
piece of information into sound to communicate with the person (speech 
synthesis, col. 4, lines 4-9). 

20. As per claim 9, Sasai teaches: 

the piece of information was entered into the assistant by the user (user 
has assistant memorize information, col. 7, lines 38-45); and 

wherein the personal assistant further includes a formatting process that 
transforms the expression input by the user into a question with one or more 
phrases corresponding to the one or more categories of information, and a 
transformation process the converts the question into an instruction comprising a 
query to the database (concept extract unit extract conceptual elements from the 
input and these conceptual elements are converted into a intention for retrieval 
by the intention interpretation unit, col. 4, line 45 to col. 5, line 15). 

21 . As per claim 1 0, Sasai teaches the piece of information to be entered 
through voice (input is speech prior to storage, Fig. 6, elements S61 and S65). 

22. As per claim 12, Sasai teaches that if the assistant cannot resolve an 
ambiguity in the expression, the personal assistant provides the person with a 
number of alternatives to resolve the ambiguity (presents plural pieces of 
information if conditions are not completely met, col. 10, lines 41-46). 

23. As per claim 26, Sasai teaches a method for obtaining information for a 
requestor interacting with a handheld computing device, said method comprising: 

storing a plurality of data items in a database organized by one or more 
categories of information (private information storage unit stores the information 
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in association with the user intention index and conceptual index hence 
categorizing the data, col. 6, lines 24-34 and Fig. 5); 

receiving an input voice expression from the requestor and converting the 
input voice expression into a text string (interfacing unit, col. 4, lines 28-33); 

processing the text string using grammatical and semantic processing to 
determine a natural language meaning for the text string (sentence structure 
analysis, semantic analysis and intention extraction unit, col. 4, lines 45-50, lines 
66-67 and col. 5, lines 1-15); 

determining a category of information corresponding to the input by the 
requestor based on one of a direct context specified by the requestor or an 
inferred context based on the expression input (concept extraction unit and 
intention interpretation unit process the text through natural language processing 
hence inferring the context of the input, col. 4, line 45 to col. 5, line 15); and 

processing the natural language meaning for the text string to extract, 
from the database, a piece of information from the category of information (the 
information management unit performs retrieval of data from private information 
storage unit, col. 5, lines 16-30). 

Sasai does not teach recognizing the requestor's voice in order to retrieve 
information personal to the requestor and only for the requestor upon recognition 
of the requestor's voice. 

Alpdemir teaches a PDA that has natural language speech recognition 
capabilities that suggests using voice recognition to authenticate a user (col. 6, 
lines 14-20). 
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It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the system of Sasai to authenticate a user for access to the 
user's personal information as taught by Alpdemir because it would make the 
user's information more secure hence making the user more likely to store the 
information on the device. 

24. As per claim 27, Sasai teaches transforming the expression input by the 
user into a question and converting the question into an instruction comprising a 
query to the database (concept extract unit extract conceptual elements from the 
input and these conceptual elements are converted into a intention for retrieval 
by the intention interpretation unit, col. 4, line 45 to col. 5, line 15). 

25. As per claim 28, Sasai teaches presenting the retrieved information to the 
requestor (output the information to the interfacing unit, col. 5, lines 16-22). 

Conclusion 

26. Applicant's amendment necessitated the new ground(s) of rejection 
presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. 
See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
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action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Matthew J Sked whose telephone number is 
(571 ) 272-7627. The examiner can normally be reached on Mon-Fri (8:00 am - 
4:30 pm). 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, David L Ometz can be reached on (571)272-7593. The 
fax phone number for the organization where this application or proceeding is 
assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 
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